Information capacity of the Hopfield model
نویسندگان
چکیده
The information capacity of general forms of memory is formalized. The number of bits of information that can be stored in the Hopfield model of associative memory is estimated. It is found that the asymptotic information capacity of a Hopfield network of N neurons is of the order N3 b. The number of arbitrary state vectors that can be made stable in a Hopfield network of N neurons is proved to be bounded above by N.
منابع مشابه
محاسبه ظرفیت شبکه عصبی هاپفیلد و ارائه روش عملی افزایش حجم حافظه
The capacity of the Hopfield model has been considered as an imortant parameter in using this model. In this paper, the Hopfield neural network is modeled as a Shannon Channel and an upperbound to its capacity is found. For achieving maximum memory, we focus on the training algorithm of the network, and prove that the capacity of the network is bounded by the maximum number of the ortho...
متن کاملComparison Between Kanerva's SDM and Hopfield-Type Neural Networks
The Sparse, Distributed Memory (SDM) model (Kanerva. 1984) is compared to Hopfield-type, neural-network models. A mathematical framework for cornporing the two models is developed, and the capacity of each model is investigated. The capacity of the SDM can be increased independent of the dimension of the stored vectors, whereas the Hopfield capacity is limited to a fraction of this dimension. T...
متن کاملCapacity for Patterns and Sequences in Kanerva's SDM as Compared to Other Associative Memory Models
The information capacity of Kanerva's Sparse, Distributed Memory (SDM) and Hopfield-type neural networks is investigated. Under the approximations used here, it is shown that tire total information stored in these systems is proportional to the number connections in the network. The proportionality constant is the same for the SDM and Hopfield-type models independent of the particular model, or...
متن کاملProbabilistic Information Capacity of Hopfield Associative Memory
T his pap er defines a form al pr obabilist ic notion for the information capac ity of the Hopfield neur al network model of associat ive memory. A mathematical express ion is derived for the number of random binary pat terns that can be stored as stable states in a Hopfield model of memory with n neur ons with a given probab ility. The derivati on is based on a new approach using two powerfu l...
متن کاملDynamic Capacity Estimation in Hopfield Networks
Understanding the memory capacity of neural networks remains a challenging problem in implementing artificial intelligence systems. In this paper, we address the notion of capacity with respect to Hopfield networks and propose a dynamic approach to monitoring a network’s capacity. We define our understanding of capacity as the maximum number of stored patterns which can be retrieved when probed...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- IEEE Trans. Information Theory
دوره 31 شماره
صفحات -
تاریخ انتشار 1985